Backpropagation without Multiplication

Hans Peter Graf AT&T Bell Laboratories Holmdel, NJ 07733 The back propagation algorithm has been modified to work without any multiplications and to tolerate comput.ations with a low resolution, which makes it. more attractive for a hardware implementatioll. Numbers are represented in float.ing point format with 1 bit mantissa and 3 bits in the exponent for the states, and 1 bit mantissa and 5 bit exponent. for the gradients, while the weights are 16 bit fixed-point numbers. In this way, all the computations can be executed with shift and add operations . Large nehvorks with over 100,000 weights were t.rained and demonstrat.ed the same performance as networks comput.ed with full precision. An estimate of a circuit implementatioll shows that a large network can be placed on a single chip , reaching more t.han 1 billion weight updat.es pel' second. A speedup is also obtained on any machine where a multiplication is slower than a shift operat.ioJl.